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Abstract 


At  a  fixed,  well-surveyed  location,  position  determinations  from  a  MAGR 
(Miniaturized  Airborne  Global  Positioning  System  Receiver)  averaged 
over  a  six-week  period  were  correct  to  within  0.5  m.  However,  the  stan¬ 
dard  deviation  of  an  individual  position  determination  was  56  m.  Almost 
20  percent  of  the  individual  position  determinations  had  errors  exceeding 
20  m.  One  in  every  300  position  determinations  had  an  error  exceeding 
0.5  km.  This  anomalously  large  error  distribution  tail  raises  questions 
about  the  MAGR's  suitability  for  some  Army-critical  functions,  such  as 
precision  guidance. 
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Introduction 


From  mid-October  1996  to  mid-January  1997,  a  Rockwell  Miniature 
Airborne  GPS  (Global  Positioning  System)  Receiver  (MAGR)  was  placed 
at  a  fixed,  well-surveyed  site:  die  NASA  Goddard  Space  Flight  Center 
(GSFC)  Geophysical  and  Astronomical  Observatory  (GGAO)  in 
Greenbelt,  Useful  data  were  recorded  during  the  six-week  period 
from  early  December  1996  to  mid-January  1997.  Continual  position 
determinations  were  recorded  at  1-s  intervals  at  every  opportunity  during 
that  period.  We  report  here  the  results  of  a  study  of  the  accuracy  of  those 
position  determinations.  In  general,  results  in  field  use  would  be  expected 
to  be  poorer  than  those  given  here,  because  of  receiver  motion  and  a 
continually  changing  environment. 

The  site  in  question  was  surveyed  on  24  May  1995,  and  placed  on  the 
WGS  84  geodetic  system.  The  surveyed  site  coordinates  (Xq,  Yq,  Zq)  are 
shown  in  table  1. 

Over  the  six-week  test  span,  the  MAGR  made  continuous  estimates 
(Xj-,  y,-,  Z,),  f  =  1, . . .,  N,  of  the  site  position  in  the  same  reference  system  at 
1-s  intervals  during  the  periods  when  it  was  active.  Because  tine  volume 
of  data  at  that  fine  time  spacing  was  quite  large,  and  because  the  indi¬ 
vidual  position  determinations  were  changing  so  little  on  that  time  scale, 
tins  study  employed  30-s  sampling  of  the  total  data  set,  except  where 
otherwise  noted.  This  left  N  =  8573  individual  sampled  position  determi¬ 
nations  on  which  to  base  our  analysis.  From  those,  we  compiled  a  number 
of  plots  and  statistics  to  illustrate  the  accuracy  of  these  measures. 


Table  1.  WGS  84 
surveyed  coordinates 
of  MAGR  at  GGAO. 


Site 

Surveyed 

coordinate 

value  (m) 

^0 

+1,130,774.372 

^0 

-4,831,255.014 

Zo 

+3,994,200.505 

1 


Mean  Position 


First,  we  formed  differences  between  the  individual  MAGR  position 
determinations  and  the  surveyed  position,  which  we  call  "position 
errors": 


AX.  =  x,-Xo,  Ay,  =  y,-yo,  az.  =  z,-Zo. 

Then  we  computed  simple  arithmetic  means  of  the  position  errors: 


The  mean  position  errors  and  their  standard  deviations  are  shown  in 
table  2.  We  note  that  the  mean  position  errors  are  zero  to  within  one 
standard  deviation,  indicating  that  the  MAGR  apparently  had  no  serious 
long-term  bias  problems  as  great  as  0.5  m.  However,  the  standard  devia¬ 
tions  are  surprisingly  large  for  such  a  large  value  of  N.  We  show  the 
reason  for  this  shortly  when  we  examine  the  detailed  behavior. 


Table  2.  Position 
errors  of  MAGR  at 
GGAO. 


Coordinate 

MAGR- 

(mean) 

WGS  84  (m) 

(st  dev) 

<AX,.> 

+0.282 

±  0.491 

(AY,) 

+0.264 

±  0.342 

(AZ,) 

-0.087 

±  0.234 

2 


(Ui)  'xv 


Individual  Position  Errors 


For  our  first  mspection  of  the  individual  position  errors,  we  simply 
plotted  AY  I  against  AX^,  as  shown  in  figure  1.  It  was  immediately  evident 
fiiat  something  serious  was  wrong,  because  errors  exceeding  30  m  ought 
to  be  rare  for  any  authorized  receiver  capable  of  10-m  or  better  accuracy. 
Yet  not  only  were  30-m  position  errors  common,  some  errors  even 
reached  the  1-km  level.  Figure  2,  which  plots  AZj-  against  AY;,  shows  tihat 
the  third  dimension  has  similar  behavior,  although  with  a  somewhat  less 
extreme  range  of  ±400  m  in  AZ,-. 

The  computed  standard  deviation  of  a  single  AXj-  value  is  45  m;  for  a 
single  AYj-  value  it  is  32  m;  and  for  a  single  AZj  value  it  is  22  m.  The  mafii- 
ematical  reason  for  these  unexpectedly  large  deviations  is  simply  that  a 
single  error  of  1000  m  contributes  more  to  the  standard  deviation  than  do 
8000  "normal"  observations  with  errors  of  order  10  m  each.  So  we  see  that 
the  error  distribution  is  so  far  from  a  normal  distribution  that  it  cannot  be 
accurately  represented  by  a  standard  deviation.  For  ttie  same  reason,  we 
do  not  attempt  to  compute  other  such  customary  error  measures,  such  as 
"circular  error  probable"  (CEP)  ^  and  its  three-dimensional  coimterpart 
"spherical  error  probable"  (SEP),  inasmuch  as  they  too  would  be 
misleading. 

Figure  3  plots  AXj  against  time,  showing  the  distribution  of  problem 
points  over  the  six-week  data  span.  Time  is  measured  in  hours  from 
December  1, 1996,  which  is  the  beginning  of  a  GPS  week.  It  is  blocked 
into  1-week  groups  over  the  6-week  span  of  data  collection,  beginning  40 
hours  into  the  first  GPS  week. 


Figure  1.  MAGR  (X,  Y)  position  errors. 
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Figure  2.  MAGR  (Y,  Z)  position  errors. 


^J.  L.  Leva,  M.  U.  de  Haag,  and  E.  D.  Van  Dyke,  1996.  In  Understanding  GPS:  Principles  and  Applications, 
E.  D.  Kaplan,  ed.,  Artech  House,  Boston,  p  281. 
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Figures.  MAGR(X,D 
position  errors. 
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Figure  4  shows  successively  closer  views  of  the  center  of  the  (AX^-,  AY^-)- 
position  error  distribution,  similar  to  figure  1,  revealing  more  details  for 
the  smaller  errors. 

For  further  study,  it  is  useful  to  compute  the  magnitude  of  the  position 
error  vector  (AX^-,  AY,-,  AZj)  at  each  observation: 

ARj  =  ^/AX?  +  Ay?  +  AZ?  . 

Figure  5  is  a  histogram  showing  the  distribution  of  ARj-  in  our  8573 
sampled  data  jx)ints.  Table  3  gives  the  statistics  that  actually  describe  the 
error  distribution,  emd  which  were  used  to  plot  the  histogram.  For  ex¬ 
ample,  1583  (18.46  percent)  of  the  8573  AR,  values  lie  in  the  range  from  10 
to  15  m. 

Note  that  56  percent  of  the  measures  lie  within  a  nominal  10-m  error 
sphere.  This  result  is  consistent  with  MAGR  error  descriptions  in  other 
reports,  which  have  led  to  such  conclusions  as  "The  probability  that  the 
error  lies  within  a  9.5-m  sphere  is  50  percent,  and  the  MAGR  therefore 
meets  specifications."  However,  as  the  table  shows,  that  statistic  is  highly 
misleading,  since  many  more  large  errors  exist  than  it  would  normally 
imply.  Because  the  error  distribution  is  not  even  approximately  a  normal 
distribution,  no  valid  conclusions  could  be  drawn  about  MAGR  position 
errors  in  general  from  any  single  summary  statistic.  A  histogram  such  as 
figure  5  would  be  needed  for  a  proper  assessment  of  reliability.  However, 
if  the  customary  error  parameters  had  been  supplemented  with  the 
standard  deviation  of  an  individual  position  error  (56  m  for  our  MAGR 
data),  appropriate  alarms  would  have  been  raised  from  the  start. 
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Figure  4.  MAGR 
(X,  Y)  position  errors 
at  successively  closer 
views. 


Figure  5.  Histogram  of 
position  errors. 


Table  3.  Error 
distribution  statistics. 


Bins  (m) 


Range  bins 
(m) 

Number  of 
position  errors 
in  bin 

Percentage 
of  total  errors 

0-1 

11 

0.13 

1-2 

49 

1.74 

2-3 

406 

4.73 

3-4 

594 

6.93 

4-5 

625 

7.29 

5-6 

679 

7.92 

6-8 

1293 

15.08 

8-10 

1040 

12.13 

10-15 

1583 

18.46 

15-20 

599 

6.98 

20-30 

512 

5.97 

30-40 

235 

2.74 

40-60 

278 

3.24 

60-80 

166 

1.94 

80-100 

111 

1.29 

100-200 

193 

2.25 

200-500 

70 

0.82 

500-1200 

29 

0.34 
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Anomalies 


Figvire  6  shows  high  time-resolution  detail  for  one  of  the  largest  position 
error  excursions.  AX^-  is  plotted  against  time  in  hours  over  a  39-minute 
span.  Points  are  at  1-s  intervals  (rather  than  the  usual  30-s  sampling),  and 
individual  blocks  are  six  minutes  wide.  (The  structure  of  other  anomalies 
is  usually  similar.)  Although  the  anomaly  is  slow  to  build  and  slow  to 
terminate,  the  sudden  jump  in  tiie  middle  takes  place  in  just  two  minutes. 
During  that  interval,  our  stationary  site  at  GSFC  would  register  a  velocity 
approaching  80  km/hr  if  MAGR  position  determinations  were  taken 
literally. 

We  define  the  fraction  of  individual  position  errors  within  a  sphere  of 
radius  R  as  «(R),  where  0  <  n(R)  <  1,  R^  =  +  Z^,  and  S(X)  is  the 

Dirac  delta  fimction; 

n(R)  =  4;rR2  dR  5(X  -  AX,)  -  AY,)  d(Z  -  AZ,)  . 

The  value  of  R  that  5delds  n(R)  =  0.5  is  the  traditional  SEP  performance 
parameter.  Figure  7  shows  a  plot  of  100  n(R),  the  percentage  of  individual 
position  errors  AR,-  wittvin  a  sphere  of  radius  R,  versus  Log^g  R-  The 
logarithm  of  radius  in  meters  was  used  so  that  errors  over  such  a  large 
range  can  be  seen  in  one  plot.  The  ordinate  is  just  tiie  percentage  of 
individual  position  errors  for  which  AR,-  <  R.  As  an  example,  the  plot 
shows  that  90  percent  of  the  individual  position  errors  have  Log^g  AR,-  < 
1.6,  which  corresponds  to  errors  less  titan  40  m;  10  percent  of  the  errors 
are  above  40  m. 


Figure  6.  MAGR  (X,  T) 
position  errors  at  high 
time  resolution. 
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Conclusions 


The  size  of  the  anomalies,  when  they  occur,  seems  fairly  random.  We  note 
that  most  short  MAGR  runs  would  show  a  much  smaller  standard  devia¬ 
tion.  The  longer  the  data  span  considered,  the  larger  the  standard  devia¬ 
tion  seems  to  become.  It  is  not  at  all  clear  that  we  have  sampled  its  worst 
behavior,  and  runs  longer  than  six  weeks  might  catch  still  poorer  perfor- 
mamce.  But  it  seems  clear  that  this  anomalously  large  error  distribution 
tail  raises  questions  about  the  MAGR's  suitability  for  some  Army-critical 
functions,  such  as  precision  guidance. 
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Discussion 


A  draft  copy  of  this  report  was  circulated  to  interested  parties.  We  have 
had  the  following  feedback. 

Rockwell  Corporation  has  written  a  white  paper  (MAGR  Technical 
Direction  Note,  June  15, 1998)  discussing  a  MAGR  anomaly  that  can  occur 
during  satellite  acquisition.  This  feature  remains  in  the  MAGR  software 
through  Link  10.  The  GDOP  (geometric  dilution  of  precision)  gives  no 
hint  of  a  problem  in  connection  with  this  type  of  anomaly.  However,  the 
anomalies  we  see  are  not  confined  to  acquisition  or  the  other  dbrcvim- 
stances  documented  by  Rockwell. 

Joe  Clifford  of  Aerospace  Corp.  notes  the  suivilarity  of  the  detailed 
anomaly  in  figure  6  to  a  "constellation  singularity,"  where  the  fourth 
satellite  being  tracked  goes  through  the  plane  of  the  other  three.  He 
wondered  if  our  MAGR  might  have  been  inadvertently  set  in  "hover 
mode"  or  some  oti\er  setting  that  freezes  the  constellation  (private  com¬ 
munication).  Brian  Baeder  of  MICOM,  who  set  up  the  MAGR  at  GSFC,  is 
quite  certain  no  such  settings  were  made,  inadvertently  or  otherwise.  We 
do  agree,  however,  that  the  signature  of  the  anomalies  is  Uke  tihat  of  a 
constellation  singularity. 

Paul  Olson  of  CECOM  writes  with  some  data  from  another  MAGR  at  the 
Starfire  Optical  Range  (SOR).  In  eight  runs  of  roughly  four  hours  length 
each,  over  a  four-day  span  in  October  to  November  1996,  he  also  reports 
one  large  anomaly  widi  a  similar  character  to  that  in  our  figure  6,  but 
cautions  that  other  sampled  data  had  50-percent  CEP  values  ranging 
between  3.9  and  8.1  m,  with  95-percent  CEP  values  between  7.0  and 
20.7  m.  He  questions  whether  tite  anomadies  can  be  as  frequent  as  we 
report.  He  further  notes  that  the  GDOP  did  deteriorate  during  the 
anomaly,  beginning  with  a  MAGR  constellation  change,  going  off-scale 
with  a  second  change,  and  then  gradually  recovering  as  die  GDOP  feU 
again  to  values  that  are  more  reasonable.  He  notes  that  good  constella¬ 
tions  were  available  to  die  MAGR  during  the  anomaly,  but  were  not  used. 
He  suggests  this  might  be  related  to  the  characteristics  of  the  Ashtech 
anterma  used  with  the  MAGR  for  their  and  our  runs,  perhaps  inhibiting 
die  MAGR  from  locking  onto  low-elevation  satellites  (private  communi¬ 
cation). 

Olson's  report  led  us  to  check  the  time  distribution  of  our  anomalies 
again.  We  noted  tiiat  we  also  had  only  one  large  anomaly  and  a  few  much 
smaller  ones  in  our  first  half-week  of  operation.  Diuing  that  period,  ovu 
50-percent  CEP  was  4.2  m,  and  oiu  95-percent  CEP  was  18  m,  figures  in 
the  range  of  those  seen  at  SOR.  Then  tihings  got  much  worse.  This  lasted 
until  an  8-day  break  over  the  Christmas-New  Year's  holidays,  following 
which  the  MAGR  was  again  well-behaved  for  several  days,  performed 
worse  for  the  next  several  days,  and  then  performed  poorly  for  almost 
every  run.  This  suggests  a  problem  that  worsens  widi  the  length  of  time 
that  the  MAGR  has  been  in  continuous  operation — a  matter  of  definite 
interest  and  concern  to  the  Army. 
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